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At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy. 
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Communication terminal handling messages including graphics. 

5 The invention relates to the transmission of graphics as a part of the 
messaging between communication terminals. 

On the Internet address http://www.kessler-design.com there is provided a 
graphic editor for generating a graphic picture on a Personal Computer in a 
1 0 predetermined formatted file, and to route this file via the Internet to a server 
where the message, based on the "Smart Messaging Concept" developed by 
the applicant is generated based on the received file. From here the message 
is routed to the cellular phone via an SMS gateway & SMS interworking 
Mobile Switching Center (MSC). 

15 

Philips Consumer Communications has exposed a new phone SAVVY™ GSM 
phone in Cannes 23. February 1999. This phone will be equipped with a 
message application using graphics. It is possible to include "clip art" like 
graphics in messages exchanged between two SAVVY™ phones. Basically 
20 this means that the "clip art" like graphics are handled as predefined 

characters and the character is transformed to a hexadecimal code in one 
phone, transferred back to the same predefined character in the receiving 
phone. 

25 An object of the invention is to provide a method for handling transmission of 
messages including graphics between communication terminals. 

This purpose is obtained by a method for handling messages transmitted 
between communication terminals via a wireless network, and comprising 
30 generation of a compound message including a text part and at least one 
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graphical icon part and transmission of the message via the wireless network. 
The handling of the compound message generation comprises steps of 
reading of a user inputted text part and converting the inputtet text into in a 
predefined message text format, adding a graphical part to the message, said 
5 graphical part includes a record for each of said at least one graphical icon 
part in a graphical format, and adding information in the message defining the 
position of said at least one graphical icon part in the text part. Hereby the 
phones vil be able to include a real graphical file into the message and the 
receiving phone will be able to decode the graphical icon without knowing the 
10 pattern in advance. 

The invention does furthermore relate to a communication terminal for 
handling messages and having a controller, a tranceiver for communicating 
with a wireless communication network, and a user interface through which 

15 the user operates the terminal, said user interface includes a display. The 
communication terminal furthermore comprises a message editor application 
by means of which the user is allowed to generate a compound message 
including a text part and at least one graphical icon part; said controller 
generates a message for transmission via said tranceiver. The message 

20 includes a text part in a predefined message text character format, a graphical 
part including a record for each of said at least one graphical icon part in a 
graphical format, and information in the message defining the position of said 
at least one graphical icon part in the text part. The graphics used in the 
message may be found in a clip art like gallery including a plurality of 

25 graphical icons, or manually entered by the user by means of a graphical 
editor. 

The invention does furthermore relate to a message format including a text 
part and at least one graphical icon part. The meassage includes a text part in 
30 a predefined message text character format, a graphical part including a 
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record for each of said at least one graphical icon part in a graphical format, 
and information in the message defining the position of said at least one 
graphical icon part in the text part. Hereby an existing text message carrying 
concept such as e.g. SMS messaging in the GSM system will be able to 
transport graphics when the receiving and transmitting terminals have 
respective graphical editors/readers for coding and decoding the messages. 

According to the preferred embodiment this information is handled by the 
Smart Messaging Concept. This concept was disclosed by the applicant at the 
Ceb'rt fair in March 1997 (Press release 20. March 1997) and discussed in 
details the document: "Smart Messaging Specification", Revision 1.0.0; 
September being available on the Internet address URL: 
www.forum.nokia.com/nf/concepts/smart/main.html. From this Internet 
address another document "Narrow-Band Sockets Specification", Revision 
1 .0, March 7, 1997 may be downloaded. This document describes the 
Narrow-Band Socket (developed by a joint effort by of Intel and Nokia) that 
enables applications to access various network data bearer services using a 
standard socket interface. 

According to the invention it is very important to improve the character of 
sending messages in a cellular phone network from being being a pure text 
messaging system to become a more sofisticated concept including graphics 
into messages and thereby attracting younger people who want to generate 
tags or symbols identifying themselves when sending messages. 

In addition to offering a huge amount of templates or predefined graphices the 
inventions will according to the preferred embodiment offer the user a built-in 
graphics composer application by means of which the user is allowed to 
create graphics himself directly on the phone. The user will also not need to 
buy a PC interface cable, tobe able to create graphics on a PC. 
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According to the invention it is essential to offer sending and receiving 
graphics directly from one phone to another. Furthermore the graphics are 
actually included into the message text. 

5 

Figure 1 schematically illustrates a preferred embodiment of a hand portable 
phone according to the invention. 

Figure 2 schematically shows the essential parts of a telephone for 
1 0 communication with a cellular or cordless network. 

Figure 3 illustrates the message reception scenario according to the invention. 

Figure 4 illustrates the message transmission scenario according to the 
15 invention. 

Figure 5 shows a display sequence of a phone receiving a message including 
graphics according to the invention. 

20 Figure 6 shows a display sequence of a phone inserting graphics into a 
message for transmission according to the Invention. 

Figure 7 shows a display of a graphical editor of a phone for manually 
entering a new graphical icon according to the invention. 

25 

Figure 8 schematically shows the essential parts of a GMS editor/reader 
according to the invention. 

Figure 1 shows a preferred embodiment of a phone according to the 
30 invention, and it will be seen that the phone, which is generally designated by 
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1 . comprises a user interface having a keypad 2, a display 3, an on/off button 
4, a speaker 5, and a microphone 6 (only openings are shown). The phone 1 
according to the preferred embodiment is adapted for communication via a 
cellular network, but could have been designed for a cordless network as well. 

According to the preferred embodiment the keypad 2 has a first group 7 of 
keys as alphanumeric keys, a soft key 8, a clear key 9, and a navigation key 
or a scroll key 10. The present functionality of the soft key 8 is shown in a 
separate field in the display 3 just above the key 8. 



Figure 2 schematically shows the most important parts of a preferred 
embodiment of the phone, said parts being essential to the understanding of 
the invention. The preferred embodiment of the phone of the invention is 
adapted for use in connection with the GSM 900MHz and GSM 1 800 MHz 
1 5 network, but, of course, the invention may also be applied in connection with 
other phone networks. The processor 18 controls the communication with the 
network via the transmitter/receiver circuit 19 and an antenna 20 that will be 
discussed in details below, 

20 The microphone 6 transforms the user's speech into the analog signals, the 
signals formed thereby are A/D converted in an A/D converter (not shown) 
before the speech is encoded in an audio part 14. The encoded speech signal 
is transferred to the processor 18, which i.a. supports the GSM terminal 
software. The processor 18 also forms the interface to the peripheral units of 

25 the apparatus, including a RAM memory 17a and a Flash ROM memory 17b, 
a SIM card 16, the display 3 and the keypad 2 (as well as data, power supply, 
etc.). The audio part 14 speech-decodes the signal, which is transferred from 
the processor 18 to the earpiece 5 via a D/A converter (not shown). 
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The ability to receive and send graphics via messages will require the 
features of storing a large amount of pre-specified graphics inside the phone 
(Clip-art / graphics templates), the ability to insert graphics into a message 
being composed, a method to browse the huge amount of graphics and a 
possibility to save received graphics, and to afterwards use them in message 
composing. According to the preferred embodiment of the invention it is 
furthermore possible to generate new graphics on the wireless communication 
terminal or the phone 1 . 

According to the preferred embodiment when the user receives a message, 
the reception will be indicated in Idle mode of the phone 1, like the first image 
of Figure 5. A header 50 includes some icons informing the user about e.g. 
the status of phone settings, time and un-read messages present in the 
phonebook. Furthermore it is seen from the first line that a new message is 
received, and the second line indicates the phone number or the name of the 
sender if this can be found in the phonebook based on the phone number of 
the sender. The next two lines indicate the start of the message. If several 
messages are received, the initial text from the newest message will be 
displayed. The functionality of the soft key 8 will advantageously be "read" as 
seen from the first display. Pressing this softkey 8 allows the user to read and 
edit the entire message as shown in the second display of Figure 5. 

In this case, the user has chosen to read the message, simply by pressing the 
soft-key 8 having the "Read" functionality. This will directly enter the message 
viewing as seen in the second display of Figure 5. Now the functionality of the 
soft key 8 changes to "option", and by pressing this key 8 the user gets direct 
access to e.g. delete, edit or forward the message. By using the scroll key 10 
the user may scroll through the message as seen in the third display.from the 
second and third display it is seen that the header 50 now identifies the 
sender of the message. Furthermore it is seen that the message includes 



7 

graphics 51 formed as two Dolphins (the trademark of the Danish operator 
Sonofon). 



Graphics can basically be of two types: "Large graphics" - as the Dolphins 
shown in the third display; and "In-line graphics" that means graphics with 
maximum height of that corresponding to the height of the text, e.g. 8 pixels. 
•In-line graphics" are placed in a text line and displayed as a text character. 
However the handling of the In-line graphics in the message will be similar to 
the handling of Large graphics. 

The fourth display shows a message including "In-line graphics" 52. The width 
of the graphics can basically be up to the width of a text line. However a 
maximum width of two-three character makes it easier to handle the message 
display. If the user (in the second, third or fourth display) now wants to save 
the graphics the graphics he can press the soft key 8 having the "Options- 
functionality and select "user graphics" in the option list displayed in the fifth 
display. 

When the "User Graphics" option is selected a list (page wise) of all the 
graphics stored in the message will be given. In the message shown in Figure 
5 second and third display only a single graphic 51 is included. The default 
options on the soft-key 8 will now allow the user to store the graphics directly 
to the "user graphics" gallery of the phone 1 for later use of the graphics. 

Figure 3 illustrates the message received scenario. In step 100 a message is 
received and in step 101 the GMS reader 30 checks whether the received 
message is in the GMS format or not. If not the GMS reader 30 just starts to 
wait for the next message. Otherwise the GMS reader 30 reads the message 
and starts to identify the text part and possible graphics in step 102. When 
this is done the GMS message is displayed for the user in step 103. in step 



8 

104 the user is allowed to store the received graphics and this is done in step 

105 upon request from the user. In step 106 the display of the messages 
remains until the user wants to close the message. 

5 In Figure 4 it is shown that the GMS editor starts in step 1 1 0 when the user 
wants to enter a GMS message. In step 111 the entered text is read and 
when the user wants to enter graphics he can indicate this in step 112. Then 
he will have the choice (in step 1 13) of entering a new graphic by means of 
using the graphical editor 35 (step 1 14) of by recalling a graphical icon stored 
10 in one of the memories 32 or 33 (Figure 8) in step 1 1 5. When the user does 
not want to enter further text or graphics he can request transmission in step 
116 whereby the GMS message is transmitted in step 117 and the application 
is closed. In stepl 16 the user will be allowed to close the application without 
transmitting the message. 

15 

Sending messages including graphics. 

The user has the ability to insert graphics into the message. The user can 
insert graphics from both a pre-defined Clip-art directory, and from the "user 
graphics" gallery. In the first display of Figure 6 the user has started to 

20 compose a message. The soft-key 8 now allows the user directly to send the 
message, but by using the scroll key 10 alternative functionalities, inluding 
insert of graphics in the option called "Insert Graphic" as seen in the second 
picture of Figure 6. By selecting the "Insert Graphics" option the user will be 
given a list of graphics groups, like: User Graphics, Animals, Business, 

25 Cartoons etc. as shown in the third display of Figure 6. 

This list will contain logical groups of graphics from the clip-art database, and 
- as the first item - the complete list of the graphics stored by the user. The list 
will also contain an item called "In-line icons", which will contain icons that can 
30 be included in the text lines, preferably having a maximum height 
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corresponding to 8 pixels. Preferably, the list will also contain CLI icons, start- 
up graphics etc. Selecting any of the items in this list, will give the user a list of 
all the graphics in the group. Selecting e.g. the item called "Animals" could 
give one of the graphics shown in the fourth or fifth display of Figure 6. 

5 Scrolling down with the scroll key 10 will show the next graphic in the 

"Animals" group. Pressing the soft-key 8 having the funtionality "Insert", will 
insert the graphic into the message text at the position of the cursor 53 (first 
display of Figure 6). The graphic is now inserted into the text, and the editor is 
ready for insertion of a new word to be added after the graphic as shown in 

10 the sixth display of Figure 6. 

According to the invention it is very important to improve the character of 
sending messages in a cellular phone network from being a pure text 
messaging system to become a more sofisticated concept including graphics 
1 5 into messages and thereby attracting younger people who want to generate 
tags or symbols identifying themselves when sending messages. 

The graphics editor basically allows the user to edit and define any pixels in 
the display. The pixel editor display is displayed in Figure 7 and each pixel in 
20 the final drawing consists of 3 * 3 pixels in the editor. The current position of 
the cursor is indicated with a hollow pixel 70. A small preview 71 of the final 
drawing can be shown in the lower right comer of the display. By using the 
soft key functionality "Options", the user can e.g. invert a specific pixel. The 
basic soft key functionalities under "Option" could allow the user to simply 
25 invert pixels individually, but functions like "Circle". "Insert Text". "Reverse all" 
will also be included. The cursor or hollow pixel 70 is moved in the display by 
means of navigation means. The navigation key 10 only allows the user to 
navigate in two directions, and therefore it is preferred to dedicate the "2", "4". 
"6" and "8" keys for moving the cursor in the four directions of the display. 
30 However an improved embodiment could include a four directional scroll key. 
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Animations. 

Like with messages and graphics, the graphical reader could offer the user a 
variety of pre-defined animations, which the user can attach to different 
5 "events" in the user interface. These events could e.g. be when entering a 
menu item, start-up animations, when an incoming call appears (for example 
CLI Group dependent), when the keyguard is activated or de-activated, etc. 

It is evident that the invention supports a scenario in which the user creates 
10 his own animations, e.g. on a Personal Computer and then downloads the 
animation to the phone, and used just like the pre-defined animations. The 
user can just send the animation to another user. 

Basically these animations are a sequence of pre-stored graphics that are 
1 5 displayed as a sequence. An animation generator has to be able to identify 
and access the individual graphics in a memory. Furthemore a set of 
parameters has to be present for the individual animation and the parameters 
must contain an identification of a start event, such as start up of the phone, 
detection of an incoming call etc.. Furthermore the refreshing time or period 
20 has to be identified i.d. the time one graphic is displayed before it is replaced 
by the next graphic. Finally the duration of the animation has to be given and 
this will typically be when the full animation has been played once, or when 
the graphics in the full animation have been played as an endless sequence 
for e.g. two or three times. 

25 

Graphical Messaging System. 

The Graphical Messaging System (GMS) is able to transport text, pictures 
and animations as one entity in messages handled by e.g. the Nokia Smart 
Messaging format. The Nokia Smart Messaging format is presently used in 
30 connection with the GSM messaging system and therefore the overall concept 
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may be regarded as well known for a man skilled in the art. The format of the 
messages including graphics transmitted from one phone to another 
according to the preferred embodiment of the invention will be described 
below with reference to Figure 4. The message format is based on the 
Nokia/Intel Narrow Band Socket (NBS) specification. This format is already 
well known and used for a number of services already implemented in a wide 
range of Nokia products, e.g. Over The Air (OTA) business cards, OTA 
ringing tone download etc. 

A GMS reader 30 is listening to a Narrow BAND Socket (NBS) port 5514 
decimal [158A hexadecimal] via the processor 18. The received message is 
displayed in the display 3 and temporarily stored in a memory 34. The reader 
30 identifies the text part and the graphics part of the message and the user is 
as mentioned above, able to store the received graphics in a "User Graphics- 
memory 32 for later user. 

It is well known to use text editors in e.g. GSM phones for generating a text 
for a message to be transmitted. A GMS editor 31 replaces this text editor and 
act fully as such one when handling text alone. The user is able to enter a text 
part by using the text entry means - preferably the alphanumeric keys, and 
when the user via the "Option" functionality in the soft key 8 starts to enter 
graphics he can get one preceded graphical icon from a memory 33 including 
a plurality of clip art like graphics - preferably included in a plurality of groups 
(animals, business, cartoons etc.) in order to ease the search for a specific 
icon. Alternatively he can enter the desired pattern by means of a graphical 
editor 35 (display is shownjn Figure 7) and store the manually entered 
graphic in a "User graphic" memory 32 and use the graphic in the GMS editor. 
The "User graphic" memory 32 includes manually entered graphics, graphics 
received in a message, or clip art like graphics amended in the graphical 
editor 35. 



12 

GMS format. 

The actual GMS format will be based on the Nokia Smart Messaging format, 
but the actual format will be as follows: 



< GMS-message> ::= <GMS-header>< GMS-ftem>* 

< GMS-header> ::= < GMS-Version> 

< GMS-Version> *0' ^Identifier for GMS version, current version is zero (0). 

<GMS-item> "00" <GMS-itern-!ength> <ISO-e859-1-char>* 

| "01" <GMS-Iten>length> <UNICODE-char>* 

| "02" <GMS-item-length> <OTA-bitrnap> 

f "03" <GMS-itenvlength> <Ringlng~tone-programnnlnoHanguage> 

I <common-hex-digit> <oonTmon-hex-<tfgit> <GSM-iten>length> 

<unspecifled-data> 

; Two hex characters who specify the GSM Hem type, if not 00... 03. 

Tokens 04 ... FF are reserved. If one of these values Is present, the application may either stop 

processing or skip that GMS-ttem.' 
<unspedfied-data> <defeult-char>" ; 'A stream of octets' 
<GMS-ltem-tength> 

<oommon-hex-digit> <common-hex-digit> 
<comrnon-hex-digit> <common-hex-digit> 
; 'Number of octets in the following field, in hexadecimal. 
Note that there are two octets per each Unicode character.' 
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CLAIMS 

1 . Method for handling messages transmitted between communication 
terminals via a wireless network, and comprising steps of: 

• generating a compound message including a text part and at least one 
graphical icon part; 

• said compound message generation includes: 

• reading of a user inputted text part and converting the inputted text into a 
predefined message text format; 

• adding a graphical part to the message, said graphical part includes a 
record for each of said at least one graphical icon part in a graphical 
format; 

• adding information in the message defining the position of said at least one 
graphical icon part in the text part; and 

• transmission of the message via the wireless network. 

2. Communication terminal for handling messages and having a controller, a 
transceiver for communicating with a wireless communication network, and a 
user interface through which the user operates the terminal, said user 
interface includes a display, and said communication terminal furthermore 
comprises: 

• a message editor application by means of which the user is allowed to 
generate a compound message including a text part and at least one 
graphical icon part; 

• said controller generates a message for being transmitted via said 
transceiver, said message includes: 

• a text part in a predefined message text character format, 

• a graphical part including a record for each of said at least one graphical 
icon part in a graphical format, and 

• information in the message defining the position of said at least one 
graphical icon part in the text part. 
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3. Communication terminal according to claim 2, wherein the message 
generated by the controller includes a header part including the position 
information. 

5 

4. Communication terminal according to claim 2 or 3 t wherein the header part 
of the message furthermore includes information about the size of the 
graphics. 

10 5. Communication terminal according to claim 2 - 4, wherein the message 

editor allows the user to copy a pre-stored graphical icon from memory means 
associated with the controller and containing a plurality of clip art like 
graphical icons. 

15 6. Communication terminal according to claim 2, and furthermore comprising 
a message reader application for automatically converting a received 
compound message into a displayable format based on the text part and the 
at least one graphical icon part 

20 7. Communication terminal according to claim 6, wherein the message reader 
application has means for allowing the user to store the at least one graphical 
icon part in memory means associated with the controller and containing a 
plurality of graphical icons. 

25 8. Communication terminal according to claim 2, wherein the message editor 
application has means for allowing the user to manually generate a graphical 
icon on the display by selectively marking dots in an icon matrix. 

9. Communication terminal according to claim 8 f wherein the message editor 
30 application has means for allowing the user to store the manually entered 
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graphical icon in memory means associated with the controller and containing 
a plurality of graphical icons. 

10. Communication terminal claim 2, wherein the message editor application 
has means for allowing the user to input a plurality of graphical icons in the 
graphical part of the message and information in the message to display this 
plurality of graphical icons as an animation sequence. 

1 1. A message format including a text part and at least one graphical icon 
part, wherein: 

• a text part in a predefined message text character format, 

. a graphical part including a record for each of said at least one graphical 

icon part in a graphical format, and 
. information in the message defining the position of said at least one 

graphical icon part in the text part. 
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